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Abstract_ 

There is a growing evidence that flipped classrooms are associated with 
increased levels of student engagement, as compared to engagement in 
“traditional” settings. Much of this research, however, occurs in post-secondary 
classrooms and is based upon self-reported engagement data. This study seeks to 
extend existing flipped classroom research by assessing behavioral engagement 
in flipped and non-flipped settings using observational instruments in three pre¬ 
college settings. Contrary to widely-reported results, this study found an increase 
in engagement in only one of the three observed settings. Analyses of the 
classroom contexts and teachers’ actions in the three settings suggests that 
student engagement is not solely a function of instructional strategy (flipped 
versus non-flipped), but is also affected by student characteristics and teachers’ 
skill and expectations. 


Introduction 

There is growing evidence that student engagement, a multi-dimensional construct that Astin (1984, p. 297) 
defines as “total amount of physical and psychological energy that the student devotes to the academic 
experience,” is directly related to student achievement (Carini, Kuh, & Klein, 2006; Furlong & Christenson, 
2008; Lazowski & Hulleman, 2016; Lee & Smith, 1993; Willingham, Pollack, & Lewis, 2002). It is for that 
reason that student engagement is, itself, becoming an objective of classroom instruction. If teachers can 
implement strategies that increase student engagement, research suggests and practitioners hope, then 
mathematics achievement will increase. 

The interest in student engagement is, perhaps, one reason for the exploding interest in the flipped classroom. 
As defined by Bergmann and Sams (2012), the flipped classroom alters the sequence of in-class and out-of-class 
activities that characterize traditional mathematics classrooms. In traditional settings, teachers review homework 
solutions and address students’ questions; present new content; model the use of the new content through 
several examples; assign homework addressing the new content; and, if any time remains, allow students to 
begin their homework. Students then return the following day with their solutions and questions to the assigned 
homework, and the cycle begins anew. In the flipped classroom, however, content that was traditionally 
introduced in class is presented to students before class via videos, simulations, or other electronic means. Thus, 
rather than being introduced to new content in the classroom, students arrive in class with some familiarity of 
the content. Classroom time is then devoted to discussions, explorations, collaboration, and additional examples 
to strengthen student understanding. For many teachers, one of the appeals of flipped instruction is the 
opportunity to engage students in a broad range of learning activities in a setting in which they, and the students’ 
peers, are readily available to assist and collaborate. 

According to numerous researchers (e.g., Deslauriers, Schelew, & Wieman, 2011; Findlay-Thompson, & 
Mombourquette, 2014; Gaughn, 2014; Hung, 2015), increased student engagement is one of the by-products of 
the flipped classroom. Yet, the limited nature of existing research suggests that the relationship between 
engagement and flipped instruction has not been conclusively established. Bormann’s (2014) recent review of 
flipped classroom research, for instance, notes the lack of pre-college research contexts and subsequent 
limitations on the generalizability of existing research findings. Although the number of studies in pre-college 
settings is increasing (Talbert, 2016), it is still true that most flipped classroom research has occurred in post¬ 
secondary settings. Additionally, studies of student engagement in flipped classrooms have, for the most part. 








Int J Educ Math Sci Technol 249 


relied upon self-report instruments and data - which are commonly utilized to assess cognitive and emotional 
engagement (Fredricks, Blumenfeld, Friedel, & Paris, 2005; Pintrich & DeGroot, 1990). Given that students 
engagement is a multi-dimensional construct (Parsons, Nuland & Parsons, 2014), however, the reliance upon 
self-report data means that little is truly known about students’ actions (i.e., behavioral engagement) in flipped 
classrooms. Moreover, a preponderance of self-report data raises some doubts about the validity of existing 
research findings. Is a student’s self-reported “engagement” actually indicative of authentic, meaningful 
engagement in the classroom? 

In this study, we address several gaps in the existing research on student engagement in a flipped classroom 
environment. In particular, this study examines the relationship between flipped classrooms and student 
engagement in three pre-college settings. In addition, this study uses triangulated, direct observations of student 
behavior (rather than self-report data) as the basis of judgements about engagement. Through direct observations 
of three teachers and their students in flipped and “traditional” instructional settings, this study examines 
whether behavioral engagement in the flipped classroom differs from that of non-flipped classrooms. While the 
reliance on observational data introduces limitations itself (e.g., the study makes no effort to assess students’ 
affective response to instruction), this study has the potential to offer insights about engagement in flipped 
classrooms that, heretofore, have yet to be fully explored. 


Review of Related Research 

In its simplest form, the flipped classroom inverts traditional approaches to classroom instruction - moving 
content instruction (via content-rich videos) from the classroom to time outside of class, and replacing in-class 
instruction with individual exploration and practice. To some, the reliance on video lectures simply perpetuates 
ineffective and outdated instructional practices (Nielsen, 2012). Yet, instruction in many flipped classrooms has 
evolved beyond the simple “video lecture at home and homework in the classroom” model to incorporate 
research-based and standards-based practices. In his reform of Middle Tennessee State University’s pre-calculus 
class, for instance, Strayer and his colleagues (Strayer, Hart, & Bleiler-Baxter, 2016) create videos that 
encourage students to complete a series of related examples and identify the patterns and generalizations 
embedded in these examples. In the classroom, then, students work collaboratively, comparing generalizations 
and seeking to verbalize, extend, and apply their mathematical discoveries. 

Recent publications from the Flipped Learning Network (FLN) reflect these expanded conceptions of flipped 
instruction. According to the FLN, a flipped classroom is one in which students view content-based videos out- 
of-class and build upon these videos through in-class instruction. Simply flipping the classroom, however, does 
not guarantee that students reap the potential benefits of the approach. The goal of the flipped classroom, 
according to the FLN (2014), is to enact flipped learning - which it defines as a “pedagogical approach in which 
direct instruction moves from the group learning space to the individual learning space, and the resulting group 
space is transformed into a dynamic, interactive learning environment where the educator guides students as 
they apply concepts and engage creatively in the subject matter.” According to the FLN and other advocates 
(Moore, Gillett, & Steele, 2014; Talbert, 2014), therefore, the underlying goal of flipped classrooms extends 
beyond simply altering the sequence and setting of instructional activities. In particular, flipped classrooms are 
viewed as tools to better engage learners. 

In the research literature, increased levels of student engagement are often associated with classrooms and 
content that are perceived as relevant (Shell & Husman, 2001), promote student-to-student interactions (Martin 
& Dowson, 2009; Wentzel, 1999), and provide students with ownership over their learning (Roeser, Eccles, & 
Sameroff, 2000). Intuitively, the flipped classroom possesses many of these characteristics. Flipped learning, as 
defined by the FLN and implemented by many teachers (e.g., Strayer, et ah, 2016), for instance, utilizes 
investigative tasks that allow students to discover ideas and concepts on their own, thus enhancing students’ 
ownership of class content. Similarly, investigative tasks and class discussions often occur in collaborative 
groups, thus promoting positive peer-to-peer interactions. It is natural to ask, therefore, whether intuitive beliefs 
about flipped classrooms as vehicles for increasing student engagement are actually true. 

In fact, there is a growing body of research that assesses the impact of flipped classrooms on student 
engagement. Lape and her colleagues (Lape, Levy, Yong, Haushalter, Eddy, & Hankel, 2014), for instance, 
reported the results of a quasi-experimental study of university students in flipped and non-flipped engineering 
and mathematics classrooms. Student self-report data revealed slightly higher levels of perceived engagement in 
traditional classrooms, as compared to the experimental flipped classrooms. These data were supported by 
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students’ short-answer responses, which revealed the perception that flipped classrooms were more distracting 
and less focused than traditional classrooms. 

Lape’s report of diminished engagement in flipped classrooms, however, contradicts the majority of published 
results (e.g.. Hung, 2015; McLean, et ah, 2016; Mclaughlin, et ah, 2014; Sun, et ah, 2016). Of these existing 
studies, the researchers of the current study consider several to be particularly informative and relevant. 
Deslauriers, Schelew, and Wieman (2011), for instance, conducted the only known observational study of 
behavioral engagement in flipped and non-flipped classrooms and serves as a model for our research design. 
Utilizing an observational protocol, pairs of observers assessed baseline levels of engagement in non-flipped 
lessons for both the control and treatment university-level physics classes, and end-of-treatment levels in 
subsequent non-flipped, control-group lessons and flipped treatment-group lessons. Although the authors 
provide no statistics, they report no increase in engagement in the control section, but a “doubling” of engaged 
students in the flipped section. The works of Bhagat, Chang, and Chang (2016) and Strohmeyer’s (2016) are 
also noteworthy because, like the current study, they examine engagement among high school mathematics 
students. Utilizing a variety of self-report methods, the authors concluded that students perceived themselves to 
be more engaged and interactive in flipped settings. Thus, there is some evidence that flipped instruction can 
enhance engagement among pre-college, mathematics students. Overall, Bormann’s (2014) review of research 
on the affordances of flipped learning includes seven studies that focused on student engagement, and all seven 
found that a majority of students (typically between 70% and 80%) perceived themselves to be more engaged in 
flipped classrooms, as compared to traditional settings. Like the majority of published works, however, none of 
these studies employed observational instruments or examined engagement in pre-college settings. 


Overview and Method 

The current, mixed-method study examines student engagement in three pre-college classrooms. Utilizing an 
observational tool, similar to that employed by Deslauriers, Schelew, and Wieman (2011), the study examines 
the behavioral engagement of students in flipped and non-flipped classrooms. The overarching research question 
is whether there is a difference in behavioral engagement, as measured by the observational instrument, in the 
two settings. The study focuses on behavioral engagement because (1) there is a documented relationship 
between engagement and student achievement, (2) this dimension of the engagement construct has not received 
significant research attention, and (3) the researchers believe that independent observational assessments of 
engagement may be more valid and reliable than self-report assessments. 


Participants and Context 

The subjects in the study were students and teachers in three pre-college mathematics classrooms - two high 
school classrooms and one seventh grade classroom. The high school classrooms included an algebra 1 class of 
27 ninth-grade students (Setting 1) and an algebra 2 class of 24 eleventh and twelfth-grade students (Setting 2). 
Settings 1 and 2 were in high schools that would both be considered suburban, although the school context for 
Setting 2 contained a slightly higher percentage of students qualifying for free and reduced lunch (40%) than 
Setting 1 (30.2%). Neither school included a significant number of students from underserved populations. The 
seventh-grade, general mathematics class of 26 students (Setting 3) was located in a rural district with a 
relatively high percentage of students qualifying for free and reduced lunch (67.7%). In addition. Setting 3 
included a small, but growing percentage (8%) of Hispanic students. 

In addition to a diversity of schools, grade levels, and students, there was diversity in the teachers involved in 
the study. The teacher in Setting 1 (Teacher 1) has a middle grades (5-9) mathematics endorsement and teaches 
freshman-level algebra and geometry courses. She has been in the classroom for 16 years and, as a recent 
finalist for the Presidential Award for Mathematics and Science Teaching, is widely recognized for her 
effectiveness. Teacher 2 has a secondary mathematics certification and 28 years of classroom experience. The 
school in Setting 2 offers three levels of algebra, geometry, and algebra 2 classes - honors, general, and 
apprentice. Apprentice-level courses are designed for students that have not met with success in school or 
mathematics. Although typically more challenging to teach than general and honors-level classes. Teacher 2 
purposely requests the apprentice-level courses. Teacher 3 has both secondary and middle-grades mathematics 
endorsements and teaches a range of seventh and eighth-grade classes in a middle grades (6-8) school setting. At 
the time of the study, she was in her sixth year of teaching. Table 1 summarizes the classroom contexts for this 
study. 
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Table 1. An Overview of the classroom contexts 


Teacher/ 

Setting 

Certification 

Experience 

Class 

Grade(s) 

1 

Middle grades mathematics 

16 years 

Algebra 

9th 

2 

Secondary mathematics 

28 years 

Algebra 2 (Apprentice) 

11th & 12th 

3 

Secondary & Middle grades 
mathematics 

6 years 

General Seventh Grade 
Mathematics 

7th 


Procedures 

During the summer prior to the study, the three teachers participated in a five-day, grant-funded, professional 
development workshop on flipped instruction. In the workshop, each teacher received technology to support the 
creation of out-of-class videos and worked with others in the workshop to develop content videos and 
accompanying classroom lessons. Teachers were granted the freedom to design materials and utilize 
instructional strategies that, in their opinion, best suited the needs of their students. In keeping with Strayer’s 
(2007) admonition that out-of-class and in-class instruction must be mutually supportive, each flipped lesson 
was to incorporate the flipped classroom “tripod:” (1) a content-rich video; (2) guided notes, discussion 
prompts, or other assessments to be completed by students during or after watching the video; and (3) classroom 
lessons or activities that build upon the content video and out-of-class assignment. As project participants, each 
teacher committed to flip one unit of one class, with the flipped unit consisting of at least 8-10 content videos. 
Although not required to do so. Teachers 1 and 3 actually flipped several units during the year. 

To compare engagement in flipped and non-flipped classrooms, one flipped and one non-flipped unit was 
selected for each teacher. The selection process was not random, but was purposeful. In particular, the 
researchers selected the first flipped unit implemented by each teacher, and a non-flipped unit that immediately 
preceded or followed the flipped unit. For Settings 1 and 3, the selected non-flipped units preceded the flipped 
units. In Setting 2, however, the observed flipped unit preceded the non-flipped unit. Two lessons were selected 
for observation and data collection in each of the flipped and non-flipped units for each teacher - one in the first 
half of the unit and another in the second half. Two observers, trained in the use of the behavioral engagement 
observation instrument, observed each of the twelve lessons. One observer was a faculty member in 
mathematics education and principal investigator of the professional development project, whereas the other 
observer was an undergraduate pre-service teacher and member of the grant evaluation team. 

To collect engagement data, the two observers each observed the same lessons. Sitting on opposite sides of the 
classroom, each observer independently selected six students in each of the observed lessons as foci for the 
observation. The observers did not select the students randomly, but sought to obtain diversity of gender, 
ethnicity (if appropriate), and location in the classroom. Prior to data collection, the students selected by each 
observer were not revealed to the other observer. 

A modified version of the instrument utilized by Deslauriers, Schelew, and Wieman (2011) was employed to 
assess behavioral engagement. This behavioral engagement observation instrument (BEOI) partitioned each of 
the observed lessons into nine, five-minute intervals. For each of the students selected by each observer, the 
nature of the observed behavior during the five-minute interval was documented with respect to four a priori 
categories of on-task behavior and five categories of off-task behavior (see Table 2). In the event that a student 
exhibited multiple on-task or off-task behaviors, or some combination of on- and off-task behavior during an 
interval, the observers used their judgement to identify the behavior exhibited during a majority of the time 
interval. In addition to classifications of student behavior, the observers recorded field notes on the instrument 
regarding the teachers’ instructional activity during each five-minute interval. To ensure reliable results, the two 
observers piloted the use of the instrument in segments of several university classrooms prior to the study. 
Comparing the observed number of students classified as engaged in each five-minute time interval, the pilot 
training achieved an inter-rater reliability (correlation) of 0.78. A copy of the BEOI is contained in the 
Appendix. 
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Table 2. Categories of on-task and off-task behavior 


Behaviors Categorized as On-Task 
Listening/Attending to Teacher 
Taking Notes 

Speaking to Teacher or Other students about 
Lesson related tasks 

Completing mathematics problems or other 
lesson-related, hands-on activities 


Behaviors Categorized as Off-Task 

• Passive, inattentive 

• Completing work for other classes/subjects 

• Speaking to others about non-mathematical 
topics or activities 

• Disruptive behavior. 

• Completing tasks unrelated to classroom lesson 
(e.g., texting) 


Data Analyses 

For each observer and for each five-minute interval, researchers counted the total number of the observed six 
students exhibiting on-task (or “engaged”) behaviors. For each time interval, the total number of engaged 
students recorded by each observer were then averaged to obtain an average engagement rating for each interval, 
under the belief that the average of the two observer scores would provide a more reliable measure of 
engagement for that time interval than the scores of the individual observers. Because each observer categorized 
the behavior of six students, the maximum possible engagement score for each five-minute rating was six, and 
the minimum possible score was zero. Since two flipped and two non-flipped lessons were observed for each 
teacher, and nine interval engagement scores were obtained for each lesson, a total of 18 flipped and 18 non- 
flipped interval engagement scores were obtained for each teacher. Mann-Whitney tests were then used to 
determine whether there was a significant difference in the median level of engagement in each teacher’s 
classroom. 

One of the challenges with interpreting flipped classroom research is the wide range of classrooms that are 
labeled as “flipped.” Likewise, there are many varieties of the “non-flipped” classroom. To better interpret 
results, therefore, the researchers recorded field notes of teacher’s actions during each five-minute interval 
(recorded on the BEOI) to categorize the teachers’ instructional focus. In particular, teachers’ actions during 
each time interval were classified as “teacher-centered” or “student-centered”. For instance, lectures or teacher- 
led discussions were considered teacher-centered actions, while independent or collaborative work on problems 
and classroom time devoted to student explorations of problems or concepts were considered student-centered 
actions. Based upon the observed teacher actions, each of the nine, five-minute intervals was labeled as 
predominantly student-centered or teacher-centered. With 18 observed intervals per teacher for each 
instructional setting, a two-proportion test enabled researchers to determine whether there was a difference in 
instructional focus (student-centered versus teacher-centered) in flipped and non-flipped settings. 


Observer 1 


Observer 2 

Teacher 1 


Teacher 1 

Flipped Lesson 1 


Flipped Lesson 1 

Engagement Score 


Engagement Score 

For Interval i 


For Interval i 





Teacher 1 
Overall Flipped 
Engagement Score 
(Average of Scores for 
Two Flipped Lessons) 


Figure 1. The data synthesis process for teacher 1, flipped instruction 
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Additional student engagement scores were calculated for each teacher. For each observed lesson, for instance, 
the nine time-interval ratings were averaged to obtain a total engagement rating for the lesson. Engagement 
ratings for the two flipped lessons were then averaged to obtain an overall flipped engagement rating for each 
teacher, and ratings for the two non-flipped lessons were averaged to obtain overall non-flipped ratings. Figure 1 
depicts an overview of the process of obtaining engagement scores for the time intervals, interval average, 
lesson average, and overall flipped engagement scores for Teacher 1. Similar processes were used to determine 
interval, interval average, lesson average, and overall flipped and non-flipped engagement scores for each of the 
three teachers. While the small number of teachers (n = 3) prevented a statistical comparison of overall 
engagement ratings in flipped versus non-flipped classrooms, these data were analyzed for qualitative 
differences. 


Results and Discussion 

Table 3 presents the overall average engagement levels of the three teachers in their flipped and non-flipped 
classrooms, and Figure 2 represents a visual display of these data. In the flipped class for Teacher 1, for 
instance, the average engagement level of 5.4 students suggests that during any given observed five-minute 
intervals, an average of 5.4 students (out of a possible 6) exhibited on-task behaviors and were categorized as 
“engaged.” Mann-Whitney tests for differences in the median engagement levels between the flipped and non- 
flipped engagement levels for each teacher yielded no significant differences for Teacher 1 and Teacher 2 (p = 
0.405 and p = 0.92, respectively), but do reveal a significant difference (p = 0.011) for Teacher 3, with median 
engagement in the flipped class exceeding that of the non-flipped class. 


Table 3. Overall flipped and non-flipped engagement levels for each teacher 


Teacher 

Flipped Class 

Non-Flipped Class 

1 

5.4 

5.6 

2 

2.55 

2.5 

3 

4.66 

3.5 



Based upon the data in Table 3 and Figure 2, the researchers adopted informal labels for the three settings: High 
Engagement (Teacherl/Setting 1), Moderate Engagement (Teacher3/Setting 3), and Low Engagement 
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(Teacher2/Setting 2). In the High Engagement setting, for instance, at least 90% of the observed students were 
classified as engaged, on average. In contrast, fewer than 50% of the observed students in the Low Engagement 
setting were classified as engaged. From the Mann-Whitney test, no significant differences were detected in 
behavioral engagement in the High Engagement and Low Engagement settings. In the Moderate Engagement 
setting, however, students exhibited greater behavioral engagement in the flipped classroom. 

To examine the extent to which distinct instructional strategies were employed in flipped and non-flipped 
classrooms, teachers’ actions during each five-minute interval were classified as predominantly student-centered 
[S] or teacher-centered [T], As an example. Table 4 and Table 5 depict the actions of Teacher 3 in one non- 
flipped and one flipped seventh-grade class, respectively, and the corresponding classifications of these actions 
for each five-minute time interval. The non-flipped class focused on developing and evaluating simple algebraic 
expressions, while the flipped class addressed solutions of one-step linear equations. 


Table 4. The actions of teacher 3 in one non-flipped lesson 


Time 

Teacher Actions 

Classification of 
Teacher Action 

11:00 

Display warm-up tasks; students working on warm-up tasks 

S 

11:05 

Discussion of solutions to warm-up task 

T 

11:10 

Discussion of solutions to warm-up task 

T 

11:15 

Displayed word problems; students worked in groups to 
develop algebraic and evaluate expressions 

S 

11:20 

Students continued to work in groups to develop algebraic 
and evaluate expressions 

S 

11:25 

Discussion of solutions to displayed problems 

T 

11:30 

Continued discussion of solutions; Present additional example 
from student textbook. 

T 

11:35 

Continued presentation and discussion of text example. 

T 

11:40 

Brief discussion of solutions to text and warm-up examples; 
distribute homework assignment. 

T 


Note that the first few minutes of each lesson were devoted to skill-based warm-up exercises and a review of 
students’ work on these tasks. Subsequently, however, the sequence of actions in Table 5 differ dramatically 
from those in Table 4. In the non-flipped class, for instance, the lesson warm-up was followed by small-group 
explorations of lesson-related problems. The remainder of the lesson, then, involved teacher-centered direct 
instruction, with Teacher 3 summarizing ideal solutions to the exploration problems and offering additional 
examples. 

In the flipped class, however, the warm-up activity was immediately followed by small-group discussion of the 
assigned content video and pre-assessments. It should be noted that students that had failed to watch the video 
or complete the pre-assessment could not participate in these group discussions, but viewed the content video on 
tablets placed near the teacher’s desk and only joined their group after completing all pre-class assignments. 
Unlike the lengthy period of direct instruction that characterized the non-flipped class, the focus of instruction in 
the flipped class shifted back-and-forth between the students and teacher - with student-centered explorations 
interrupted by brief teacher-centered commentaries. Teacher 3 contributed to the discussion, therefore, but used 
student responses to highlight the intended learning targets and continually returned the focus of instruction to 
the student groups. 

Overall, the descriptions in Table 4 and Table 5 (and similar descriptions of the actions of Teachers 1 and 3) 
serve as broad validations of the fidelity of implementation. At least fifty percent of the actions observed in each 
of the non-flipped lessons were classified as teacher centered. On the other hand, the actions in each of the 
observed flipped lessons were predominantly student-centered and utilized some version of the flipped 
classroom tripod (video, out-of-class activity or assignment associated with the video, and classroom lessons 
associated with the video). 
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Table 5. The actions of teacher 3 in one flipped lesson 


Time 

Teacher Actions 

Classification of 
Teacher Action 

11 

00 

Students complete warm-up tasks. 

S 

11 

05 

Brief explanation of solutions to warm-up task. 

T 

11 

10 

Student discussion of content video in groups 

S 

11 

15 

Continued group discussion of content video. 

S 

11 

20 

Brief student-led, whole-class discussion of content video; 
teacher summary of discussion. 

T 

11:25 

Student groups complete examples related to content video. 

S 

11:30 

Continued work on group tasks 

S 

11:35 

Continue work on group tasks. 

s 

11:40 

Brief teacher-led summary of solutions and strategies for group 
tasks; preview of subsequent content video (through an 
additional example). 

T 


Although sample size restrictions prevent a statistical comparison of strategies for each teacher. Figure 3 depicts 
the overall proportion of teacher- and student-centered actions in flipped and non-flipped settings. In non- 
flipped classrooms, 19 of the 54 (35.2%) observed teaching intervals were classified as student-centered, 
whereas 43 of the 54 (79.6%) intervals observed in the flipped classroom were classified as student centered. A 
test of the proportion of student-centered actions in the two settings revealed a significant difference (P < 
0.0001), with a greater proportion of student-centered instmctional actions in the flipped classrooms. 


Non-Flipped Classroom 



Flipped Classroom 



Student- 
Centered, 
43 , 80 % 


Figure 3. Counts/percentages of teacher- and student-centered actions 


Research conducted in university classrooms generally report higher levels of engagement in flipped 
classrooms, as compared to non-flipped classrooms (Bormann, 2014). Much of this research, however, is based 
upon students’ self-reported engagement. To our knowledge, only one publication reports the results of 
observational studies of student engagement (Deslauriers, Schelew, & Wieman, 2011), and these results are of 
university physics students. The current study examines student behavioral engagement via direct observations 
and in pre-college mathematics settings. In these regards, therefore, the current study is somewhat unique. 
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Moreover, the current study incorporates greater controls than many of the existing studies, utilizing the same 
students and teachers in flipped and non-flipped settings as the basis for comparison. 

Contrary to most existing studies (Hung, 2015; McLean, et al., 2016; Mclaughlin, et al., 2014; Sun, et al., 2016), 
however, the current study did not find higher levels of student engagement in flipped classrooms, as compared 
to non-flipped settings. In two of the three observed settings, for instance, there was no significant difference in 
behavioral engagement in the flipped and non-flipped classrooms. The flipped classroom, however, was 
associated with significant increases in behavioral engagement in one setting. The lack of differences in 
observed engagement is somewhat surprising, given that a significantly higher proportion of instruction was 
categorized as “student-centered” in the flipped classrooms, as compared to non-flipped classrooms. Our results, 
then, also appear to contradict universally-held beliefs that student-centered instruction is associated with higher 
levels of engagement (Martin & Dowson, 2009; Wentzel, 1999). 

Recent research suggests that teacher behaviors and skill can facilitate students’ engagement (Dopson, 2010). 
While the fact that student engagement increased in only one setting, despite the use of flipped instruction and 
overall increases in student-centered instruction, we believe that knowledge of the teachers in each setting - and 
careful descriptions of their instructional actions - facilitate greater understanding and more meaningful 
interpretations of these results. The fact that no significant difference was observed in the High Engagement 
setting, for instance, is probably due to the ceiling effect and the overall effectiveness of the teacher. With a 
maximum engagement rating of six, which would indicate that all six of the students observed by each observer 
were classified as engaged during each five-minute interval, there is little room for improvement from the initial 
(non-flipped) rating of 5.6. We consider the high engagement scores in this setting, therefore, to be a reflection 
of a highly qualified and effective teacher. Likewise, the lack of differences in the Low Engagement setting may 
also be attributed to contextual characteristics. According to Teacher 2, the majority of the students in the Low 
Engagement setting (apprentice-level eleventh and twelfth-grade students in an Algebra 2 class that is required 
for graduation) are not college-intending, nor have they displayed an affinity for mathematics. Moreover, in our 
observations of the setting, the teacher seemed to direct her instruction and attention toward those displaying 
“engaged” behavior, while ignoring the non-disruptive behavior of off-task students. With this combination of 
students and teacher, it is not surprising that there were low levels of engagement in both flipped and non- 
flipped classrooms. 

In the Moderate Engagement setting, however, higher levels of engagement were observed in the flipped 
classroom. Although this result is based upon a limited number of flipped and non-flipped observations, 
contextual characteristics of the setting could yield important clues regarding the observed engagement 
differences. While not the local honors class (pre-algebra), for instance. Teacher 3 described her students as 
enthusiastic, inquisitive, and “open to new ways of doing business in the classroom.” Moreover, Teacher 3’s 
organization of the flipped classroom seemed to invite greater student responsibility and involvement. Both of 
her non-flipped lessons, for instance, featured teacher-led discussions and modeling of the desired skills. In her 
flipped lessons, however, students summarized the lesson video themselves and solidified their understanding 
through collaborative student-centered explorations. 

Research suggests that student engagement is directly related to students’ ownership of the class content and 
learning (Roeser, Eccles, & Sameroff, 2000). Given this established finding, it is not surprising that engagement 
in flipped classes in the Moderate Engagement setting was greater than in the non-flipped classes. Teacher 3 
employed a greater proportion of student-centered actions in her flipped lessons (14 out of 18, as opposed to 8 
out of 18) and established a classroom culture in which assuming responsibility for one’s learning was the norm. 
Teacher 2, it should be noted, also employed a greater proportion of student-centered actions in her flipped 
sessions (14 out of 18, as opposed to 5 out of 18), yet no significant difference in engagement was observed in 
the Low Engagement setting. This result suggests that student-centeredness alone may not determine 
engagement, but that engagement is a function of many factors. Teacher 3, for instance, clearly conveyed the 
expectation that students were to complete the assigned work and participate in class. Teacher 2, on the other 
hand, focused on the engaged students, but did not endeavor to establish the same culture of responsibility. 
Thus, the teachers’ actions could explain the differences observed in the Moderate and Low Engagement 
settings. Similarly, student characteristics could have an impact. The students in the Moderate Engagement 
setting, for instance, were “enthusiastic and inquisitive,” whereas those in the Low Engagement setting were 
described as “not college intending,” and “struggling to see the relevance of algebra.” For this latter group of 
students, who may not perceive academic goals to be relevant to their future needs, an offer of greater 
ownership of an outcome that is not desired would not be expected to enhance engagement. 
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Conclusions 

There is widespread interest in the flipped classroom as a potential tool for enhancing student achievement 
(Moore, Gillett, & Steele, 2014; Strayer, Hart, & Bleiler-Baxter, 2016; Toppo, 2011). While the current study 
did not examine student achievement, the results of this study suggest that flipped classrooms may impact 
student engagement - which many link to achievement (e.g., Carini, Kuh, & Klein, 2006; Furlong & 
Christenson, 2008; Lazowski & Hulleman, 2016). As suggested by numerous research articles and confirmed 
by our research, however, factors affecting student engagement in both flipped and non-flipped classes are 
numerous and varied (Fredricks, Blumenfeld, & Paris, 2004). In particular, our research suggests that student 
engagement is not simply a function of instructional strategy (flipped versus non-flipped), but is a complex 
combination of instructional strategy, teacher abilities and actions, and student characteristics. Accurate 
interpretations of the learning and behavioral outcomes in flipped (and non-flipped) classrooms, therefore, 
requires careful consideration of the setting and the skill and actions of individual teachers. 


Recommendations 

First and foremost, our research suggests that those examining the impacts of flipped instruction cannot simply 
vary the instructional approach (flipped versus non-flipped), but must also consider the actions of the teacher, 
characteristics of the students, and other contextual factors. These descriptions of the contexts will serve two 
purposes. First, as in the present study, a consideration of context can lead to improved understandings of the 
results. Second, rich descriptions of context can lead to more detailed classifications of the broad body of 
flipped research. Even in our study, there was some variation in the flipped (and non-flipped) classes. With 
improved descriptions of the research settings, researchers might find that there are distinct subcategories of 
flipped classrooms, and that the results vary by instructional strategy. 

Although the purpose of our study was not to ascertain best practices in the flipped classrooms, our observations 
of three pre-college teachers also yielded some generalizations which could be of use to practitioners. In 
particular. Teachers 1 and 2 - both of whom experienced relatively high levels of behavioral engagement in 
their flipped classrooms - shared a few common practices. First, both maintained high expectations for 
completion of video-related assignments. Students were simply expected to view the assigned video and 
complete all ancillary assignments. Second, both encouraged active student-to-student communication and 
collaboration about the content of the video and ancillary materials in the classroom. In both classrooms, 
students communicated with an about mathematics, resulting in high levels of behavioral engagement. 
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Appendix. Behavioral Engagement Observation Instrument (BEOI) 
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BEOI Codes 


On Task: Off task: Other Observations: 


Nl: 

Listening/Attending to Teacher 

FI: 

Passive, inattentive 

N2 

Taking Notes 

F2 

Completing work for other 
classes or subjects 

N3 

Speaking to Teacher or Other 

F3 

Speaking to others about 


students about Lesson related tasks 


non-mathematical topics or 
activities 

N4 

Completing mathematics problems 
or other lesson-related, hands-on 

F4 

Disruptive behavior. 


activities 

F5 

Completing tasks unrelated 
to classroom lesson (e.g., 
texting) 



